Bootstrapping a System for Phoneme Recognition and Keyword Spotting in Unaccompanied Singing

Speech recognition in singing is still a largely unsolved problem. Acoustic models trained on speech usually produce unsatisfactory results when used for phoneme recognition in singing. On the flipside, there is no phonetically annotated singing data set that could be used to train more accurate acoustic models for this task. In this paper, we attempt to solve this problem using the DAMP data set which contains a large number of recordings of amateur singing in good quality. We first align them to the matching textual lyrics using an acoustic model trained on speech. We then use the resulting phoneme alignment to train new acoustic models using only subsets of the DAMP singing data. These models are then tested for phoneme recognition and, on top of that, keyword spotting. Evaluation is performed for different subsets of DAMP and for an unrelated set of the vocal tracks of commercial pop songs. Results are compared to those obtained with acoustic models trained on the TIMIT speech data set and on a version of TIMIT augmented for singing. Our new approach shows significant improvements over both.